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Abstract Body 


Background / Context: 

The gap among ethnicities and gender in mathematics achievement is a well-known 
problem. While the gap has been shrinking over the past three decades, it has not completely 
diminished (Jencks & Phillips, 1998; McGraw, Lubienski, & Strutchens, 2006). Generally 
speaking, the test scores in math of African American students have lagged behind those of white 
students. And the gender gap favors male students over female students. However, a more 
accurate description often mixes the effect of ethnicity and gender (Catsambis, 1994; Kao & 
Thompson, 2003; Riegle-Crumb, C., 2006; McGraw et ah, 2006; Else-Quest, N. M., Mineo, C. 
C., & Higgins, A., 2013). A study of 8* grade students showed that the gender difference was the 
smallest among African Americans when compared to Latino and White students (Catsambis, 
1994). McGraw and his colleagues (2006) also indicated that the male students’ advantage in 
math mostly occurred in White students and does not exist among African American students. 
Else-Quest et al. (2013) reported that, for 10* grade students, the females and males earn similar 
grades in math and science independent of ethnic group (White, African American, 

Latino/Latina, and Asian American). The differences of gender favoring males occurred in the 
level of confidence and the expectations for success. 

Many attempts to diminish the achievement gap among ethnicities and genders have been 
attempted, but have not fully succeeded. However, the development of new technology provides 
a new opportunity for decreasing these gaps. The ALEKS, Assessment and LEaming in 
Knowledge Spaces, tutoring system is one promising example of a technology that can target this 
problem. 

ALEKS is a Web-based intelligent tutoring system (ITS) that instructs students on the 
mathematical topics that they are most ready to learn. The system design was motivated by 
research at New York University and the University of California, Irvine, by a team of software 
engineers, mathematicians, and cognitive scientists with support from a National Science 
Eoundation grant. ALEKS has an artificial intelligence engine that assesses each student 
individually and continuously at a fine-grain rather than a course-grain level. It assesses students’ 
current knowledge, instructs students on the topics they are most ready to learn, and evaluates 
student performance on problems related to those topics. 

Hu and colleagues (2013) reported that intelligent tutoring system (ITS) can help reduce 
the gap between white and African American students. They (Hu et al, 2013) introduced ALEKS 
into undergraduate behavioral statistics classes. Some classes used the ALEKS system while the 
other classes used traditional lecture based classes. The results showed that the performance of 
African American students was slightly better than White students in the ALEKS condition, 
while Whites performed significantly better than African Americans in the teacher condition. A 
similar pattern of data have been found for 6* grade students in Mathematics in which large 
achievement gaps were observed in favor of White students over African-American students at 
pretest, but the achievement gap disappeared at posttest after using ALEKS (Cheney et al., 

2011 ). 

In order to better understand the role that technology can play in decreasing achievement 
gaps for gender and ethnicity, we examined the effect of interacting with ALEKS in the 6* grade 
students’ math after-school program. The prior study (Hu et al, 2013) emphasized the importance 
of considering student performance prior to the course when studying ethnic gaps (e.g. ACT 
Math and Cumulative GPA). Therefore, we use the 5* grade score of an annual state test, the 
Tennessee Comprehensive Assessment Program (TCAP), to represent the students’ math 
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performance before the intervention. Also, we used the 6* grade TCAP score as the after- 
intervention math performance. 

Therefore, the purpose of this study was to clarify the effectiveness of intelligent tutoring 
system to reduce the achievement gap among ethnicities and genders in an after-school math 
program for 6* grade students. Students were self-selected into the program. Then they were 
randomly assigned into either the ALEKS condition or second teacher condition that receives 
traditional style instruction from human teachers. This is a multiyear study. So far overall results 
for year one has shown that there was not a significant difference between the ALEKS condition 
and the control taught by human teachers, but the mathematics scores for students in the program 
was significantly higher than students not in the program (Hu et ah, 2012). Similarly for the 
second year, there was no different observed between conditions. However, students in the 
ALEKS condition required less assistance and supervision (Craig et ah, 2013). 

Purpose / Objective / Research Question / Focus of Study: 

This study aims to test how the intelligent tutoring system, ALEKS, can be used to reduce 
the gap of student mathematics achievement among ethnicities and genders in an after-school 
setting for 6* grade students. ALEKS founded on the idea that students should learn material as 
they are ready for it. ALEKS adaptively selects the next skill for a student to work on. The 
system attempts to fill learning deficits and correct misconceptions adaptively and dynamically 
using Knowledge Space Theory (Hu et ah, 2012). It tracks the knowledge states of learners in 
fine detail and adaptively responds with assignments that are sensitive to these knowledge states. 

To do so, we implement a randomized experiment in which students are either assigned to 
use ALEKS or are assigned to be taught structured lesson plans from a teacher. As such, we aim 
to answer the research question: Does ALEKS reduce the math achievement gap between two 
ethnicities (White and African American) comparing to a teacher? 

Setting: 

The experiment is conducted as an after-school program at five secondary schools in a 
school district in west Tennessee. The district serves both a midsized city and the surrounding 
county serving a total 13,607 students in grades Pre-K-12 distributed among 28 schools. The 
school system serves a largely economically disadvantaged population (68.2%) and a majority 
minority student enrollment (56.3% African American, 3.4% Hispanic, 39.3% White, and 1% 
other). 

Population / Participants / Subjects: 

Previous study has shown that sufficient time in the program is needed before it becomes 
effective (Craig et ah, 2011). So, this study focused on 102 students with excellent attendance 
(45 out of 50 days). This left the teacher condition with 22 males, 29 females; 1 1 white students 
and 40 African American students. The ALEKS condition had 28 males, 23 females; 11 white 
students and 40 African American students. This group comes from a total of 776 participants 
volunteered for our after-school program at the academic year 2010-201 1 and 201 1-2012. 

Intervention / Program / Practice: 

Students who self-selected to the after-school program were randomly assigned to one of 
two conditions (ALEKS & Teacher). Students in both conditions attended the program two days 
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a week for two hours each day. The program was conducted for 25 weeks with 40 days of 
instruction and 10 days of review assessments. 

Each two-hour session was divided into five 20-minute segments with 10-minute periods 
for start-up and dismissal. The students received three 20-minute instruction sessions. The 
instruction sessions were separated by two 20-minute down-time sessions during which students 
received snacks (E' break) and played games (2"‘^ break). 

In the ALEKS condition, students interacted with the program during each of the 20- 
minute instructional sessions. The program gave the problems appropriate to their current 
knowledge space. Upon mastery of a topic, the student is given instruction for the next topic in 
that area. The student was able to select this topic from a list of problems selected based on their 
current knowledge space (See Eigure 1). 

In the teacher condition, each of the 20-minute sessions was highly structured. The 
material to be covered was selected prior to the start of the program from the State Performance 
Indicators (SPI) topic(s). In particular, the SPIs selected were those that students typically 
missed and best would align with the needs of the student population. Members of the research 
team designed very structured lesson plans for each day of instruction. Eor example, during the 
first instructional 20-minute session teachers gave a short lecture and a demonstration of how to 
work through problems. During the second 20-minute session, the class worked as a group and 
with the teacher to complete a set of problems going over the material discussed while during the 
third 20-minute session, students worked on their own. 

Eor both the AEEKS and teacher conditions the outcome measure of performance was 
the Tennessee Comprehensive Assessment Program (TCAP). This assessment is given at the end 
of each year for grades 3-8 to all students in Tennessee. It is the test used to evaluate the level at 
which each student has learned the SPIs for that year. We use the TCAP normal curve equivalent 
score, which is the adjusted scores that gives the student relative rank among all Tennessee 
students. The scores of the 5* grade TCAP were used to assess students’ pre -program 
mathematics achievement whereas the scores of the 6* grade TCAP were used as the posttest. 

Research Design: 

This study used a randomized experiment with treatment and control group to gauge and 
compare the effect of AEEKS software to a human teacher in reducing achievement gap in 
mathematics. As stated above, students who volunteered for the study were then subsequently 
randomly assigned to the AEEKS condition or the teacher condition. 

Data Collection and Analysis: 

Data were collected through the district as well as through the program itself. The district 
provided background characteristics (gender and racial/ethnic background) of each student as 
well as student TCAP scores from both 5th grade (before the program) and 6th grade (after the 
program). The program collected information as to how often students attended. Each teacher in 
the teacher condition and each monitor in the AEEKS condition kept student role throughout the 
duration of the program. 

We used student performance on the 6th grade TCAP as a dependent variable looking for 
differences between Condition, Gender, and Ethnicity while using 5‘^ grade TCAP scores for 
mathematics as a covariate. 

Findings / Results: 
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The study was done at five schools in the same school district. An ANOVA showed that 
the schools did not have significant impact on the pretest score grade TCAP), F(4, 98) =.219, 
p = .927. 

The 5* grade TCAP scores of the students in the teacher condition did not indicate any 
significance among genders and ethnicities. In the ALEKS condition, there was an marginally 
significant 2 (Ethnicity: African American; White) X 2 (Gender: Male; Eemale) interaction, E(l, 
48) = 3.582, p = .064; rj = .069. Eollow up analyses showed a difference between White male 
and African-American male, F(l, 27) = 3.117,p = .089; = .103. Means and standard 

deviations for 5* grade TCAP scores can be found in Table 1. 

A 2 (Condition: Teacher; AEEKS) X 2 (Gender: Male; Eemale) X 2 (Ethnicity: African 
American; White) ANCOVA was conducted on student’s 6th grade TCAP scores using student’s 
5th grade TCAP scores as the covariate. This analysis indicated a significant three-way 
interaction, F(l, 93) = 5.354, p = .023; = .054. Means and standard deviations for the 6* grade 

TCAP scores adjusted by 5* grade TCAP scores can be found in Table 1. Eollow up analyses 
found that there were no significant differences among groups in the AEEKS condition while 
there was an marginally significant difference on a 2 (Gender: Male; Eemale) X 2 (Ethnicity: 
African American; White) ANCOVA for students in the teacher condition, F(l, 46) = 3.094, p = 
.085, r] = .063. The result indicated a tendency that white males underperformed in the teacher 
condition while they were the top performer in the AEEKS condition. However, the result needs 
replication since there may a sample bias for a limited number of white participants. 

It should be noticed that, in the AEEKS condition, there was an achievement gap between 
white males and African American males at the 5* grade. After interacting with AEEKS for the 
complete program, the African-American males’ achievement gap with other students has been 
narrowed. The marginally significant effect disappeared. ALEKS ’s ability to reduce African- 
American students’ achievement gap has been found by Cheney et al.’s (201 1). One reason could 
be the ability difference of the ALEKS and teachers in fulfilling of the individual needs of the 
male students. ALEKS adaptively responds learners with assignments that a sensitive to their 
ready-to-leam knowledge states. However, the individual needs of the lowest and highest 
performers may be ignored by the teacher while the majority of the students in the classroom are 
the average performers. 

Conclusions: 

It appears that the ITS (ALEKS) helps reducing the achievement gap between ethnicities. 
While more research is needed, it would appear that the ALEKS condition was able to offer 
equivalent help to students with different levels of performances, while the teachers’ help tends 
to average performers. 
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Figure 1. Screenshot of the ALEKS problem selection interface 


Table 1. 

The means and standard deviation on 5* grade and adjusted 6^’’ grade TCAP scores for all cells 
for Gender, Ethnicity, and condition. This table also includes the number of students that 
attended more than 45 days. 


Teacher condition 
Grade TCAP 6* Grade TCAP 




M 

SD 

M 

SD 


White 

Male 

47.25 

24.905 

38.89 

24.91 

4 


Eemale 

55.43 

21.31 

51.77 

21.31 

7 

African- 

Male 

48.44 

14.14 

48.50 

14.14 

18 

American 

Eemale 

43.27 

17.90 

44.55 

17.897 

22 





AEEKS condition 




5'^^ Grade TCAP 

6* Grade TCAP 

n 



M 

SD 

M 

SD 


White 

Male 

59.80 

15.09 

54.93 

15.09 

5 


Eemale 

41.83 

19.51 

43.71 

19.51 

6 

African- 

Male 

39.91 

14.10 

43.94 

14.10 

23 

American 

Eemale 

47.12 

16.77 

45.83 

16.766 

17 
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